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(57) Abstract: A first image (or other pattern) is represented 
by a first ordered set of elements A each having a value and a 
second pattern is represented by a second such set. A compar- 
ison of the two involves performing, for each of a plurality of 
elements x of the first ordered set the steps of selecting (103) 
from the first ordered set a plurality of elements x' in the vicin- 
ity of the element x under consideration, selecting (102) an ele- 
ment y of the second ordered set and comparing the elements x' 
of the first ordered set with elements y' of the second ordered 
set (each of which has the same position relative to the selected 
element y* of the second ordered set as a respective one x' of 
the selected plurality of elements of the first ordered set has rel- 
ative to the element x under consideration). The comparison it- 
self 106 comprises comparing the value of each of the selected 
plurality of elements x' of the first set with the value of the 
correspondingly positioned element y' of the like plurality of 
elements of the second set in accordance with a predetermined 
match criterion to produce a decision that the plurality of ele- 
ments of the first ordered set matches the plurality of elements 
of the second ordered set. The comparison is them repeated 
with a fresh selection of the plurality of elements x* of the first 
set and/or a fresh selection of an element y of the second or- 
dered set generating a similarity measure V as a function of the 
number of matches. Preferably, following a comparison result- 
ing in a match decision, the next comparison is performed with 
a fresh selection of the plurality of elements x' of the first set 
and the same selection of an element y of the second set. 
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Comparing Patterns 



The present invention is concerned with comparing one pattern with another, and is of 
particular interest in the comparison of two-dimensional patterns such as visual images, although 
5 applicable also to one-dimensional patterns and patterns having three or more dimensions. Standard 
approaches to pattern recognition use templates to recognise and categorise patterns [1]. Such 
templates take many forms but they are normally produced by a statistical analysis of training data 
and matched with unseen data using a similarity measure [2], The statistical analysis is normally 
carried out over a number of intuitively selected features that appear to satisfy the needs of the 

1 0 recognition task. For example, in speech recognition templates can be encapsulated as Hidden 
Markov Models derived in the frequency domain and in Optical Character Recognition the 
templates take the form of the character fonts themselves. In the case of face recognition a number 
of intuitively chosen features such as skin texture, skin colour and facial feature registration are 
used to define face templates. In a CCTV surveillance application intruders are normally detected 

15 through a process of frame subtraction and background template modelling which detects 
movement and removes background effects from the processing [3]. In many cases the number of 
features leads to a computationally unmanageable process and Principal Components Analysis and 
other techniques are used to scale down the problem without significantly reducing performance 
rhttp://www .partek.com/index.html1 . These approaches achieve great success in non-noisy 

20 environments but fail when the pattern variability and number of pattern classes increase. 

Some techniques for analysis of images or other patterns where the pattern is compared 
with other parts of the same pattern are described in our earlier patent applications as follows. 
European patent application 00301262.2 (publication No. 1 126411) (applicants ref. A25904EP#); 
International patent application PCT/GBO 1/00504 (publication No. WO 01/61648) (applicants ref. 
25 A25904WO); 

International patent application PCT/GBO 1/03 802 (publication No. WO02/21446) (applicants ref. 
A25055WO); 

U.S patent application 977,263/09 filed 16 October 2001 (publication No. 20020081033) 
(applicants ref. A25904US1); 
30 - as well as the following papers published by the inventor: 

Stentiford F W M, "An estimator for visual attention through competitive novelty with application 
to image compression", Proc. Picture Coding Symposium 2001, Seoul, 25 - 27 April, pp 101 - 104, 
2001. 
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Stentiford FWM, "An evolutionary programming approach to the simulation of visual attention", 
Proc. Congress on Evolutionary Computation 2001, Seoul, pp 851 - 858, 27 - 30 May, 2001. 

According to one aspect of the present invention there is provided a method of comparing a 
first pattern represented by a first ordered set of elements each having a value with a second pattern 
5 represented by a second ordered set of element each having a value, comprising performing, for 
each of a plurality of elements of the first ordered set the steps of: 

(i) selecting from the first ordered set a plurality of elements in the vicinity of the element 
under consideration, the selected elements having, within the ordered set, respective mutually 
different positions relative to the element under consideration; 

1 0 (ii) selecting an element of the second ordered set; 

(iii) comparing the selected plurality of elements of the first ordered set with a like 
plurality of elements of the second ordered set each of which has within the second ordered set the 
same position relative to the selected element of the second ordered set as a respective one of the 
selected plurality of elements of the first ordered set has relative to the element under 
1 5 consideration, said comparison comprising comparing the value of each of the selected plurality of 
elements of the first ordered set with the value of the correspondingly positioned element of the 
like plurality of elements of the second ordered set in accordance with a predetermined match 
criterion to produce a decision that the plurality of elements of the first ordered set matches the 
plurality of elements of the second ordered set; 

20 (iv) repeating said comparison with a fresh selection of the plurality of elements of the first 

ordered set and/or a fresh selection of an element of the second ordered set; and 

(v) generating for the element under consideration a similarity measure as a function of the 
number of comparisons for which the comparison indicates a match. 

Other aspects of the invention are defined in the claims. 

25 Some embodiments of the present invention will now be described with reference to the 

accompanying drawings, in which: 

Figure 1 is a block diagram of an apparatus for performing the invention; 
Figure 2 is a diagram illustrating operation of the invention; 

Figure 3 is a flowchart of the steps to be performed by the apparatus of Figure 1 in 
30 accordance with one embodiment of the invention; and 

Figures 4 to 9 illustrates some images and numerical results obtained for them 
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Figure 1 shows an apparatus consisting of a general purpose computer programmed to 
perform image analysis according to a first embodiment of the invention. It has a bus 1, to which 
are connected a central processing unit 2, a visual display 3, a keyboard 4, a scanner 5 (or other 
device, not shown) for input of images, and a memory 6. 

5 In the memory 6 are stored an operating system 601, a program 602 for performing the 

image analysis, and storage areas 603, 604 for storing two images, referred to as image A and 
image B. Each image is stored as a two-dimensional array of values, each value representing the 
brightness of a picture element within the array. 

The image arrays are shown schematically in Figures 2a and 2b. Image A consists of a 20 
10 x 20 array of picture elements x =( JCi, x 2 ) where x } and x 2 are the horizontal and vertical positions 
of the elements within the image. Each element has a brightness represented by a respective value 
a = a (x). Similarly, the image B consists of a 20 x 20 array of picture elements y = (y u y 2 ) having 
brightness values Z?. 

A method of image analysis performed by the program 602 is performed in accordance 
1 5 with the flowchart shown in Figure 3. It has as its aim to generate a measure V of similarity 
between the images; more specifically, a high value of V indicates that image B contains visual 
material similar to the content of image A. 

It is assumed that the images have width xmaxi and ymax\ and height xmax 2 and ymax 2 , 
and the picture element co-ordinates are X\ = 0 . . . xmaxx-l, x 2 = 0 . . . xmax 2 -l . 

20 In Step 100, a picture element x in image A is selected. They can be dealt with in any 

order, but it is convenient to select initially the element x = (e,s) and on subsequent iterations, 
successive elements in raster-scan fashion, until all have been dealt with, except for those within e 
of the edge of the image i.e. up to (xma* r e-l, xmax 2 -s~\). 

In Step 101, the score Kand a counter tries are set to zero. 

25 In Step 102, an element y = (y u y 2 ) is chosen at random from image B, within a maximum 

distance s of the position of x, that is 

|jc # — j/ f J^s for all i 

This limitation, which is optional, is represented in Figure 2b by the dotted square S. It 
improves efficiency if it is known that corresponding objects in the two images will not be shifted 
30 in position by more than s picture elements in either co-ordinate direction: in effect, s represents the 
maximum misregistration or local distortion between the two images. 
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The selection of j is also subject to the restriction that it is located no more than e from the 
edge of the image, i.e. 

s<y t < ymBXi-e-l fori= 1,2 

where ymax\ is the width or height of the image in picture elements (20 in this example). 

5 In Step 103, a set S x of picture elements x 1 , are selected, comparing the element plus at least 

one further element, m in number, randomly selected from image A, in the neighbourhood N of 
picture element x, where the neighbourhood N is a 2e+l x 2e+l square centred on x. That is, an 
element x ! = (x\\ x* 2 ) lies within N if 

|x/-x.|<£ for all i 

10 We prefer that the neighbourhood set S x includes the element x itself, although this is not 

actually essential. A neighbourhood set S y is then defined in image B, with m+l elements y'= (jV, 
y 2 ') each of which has the same position relative to element y as the corresponding element x' of S x 
has from x, that is 

y t % -yi = x i '-x i for alii 

1 5 (The reason for the edge restrictions mentioned above is of course to avoid choosing a 

neighbourhood which extends outside the image area). In Step 104, the counter tries is 
incremented, and in Step 105 its value is checked so see whether sufficient comparisons have been 
performed. 

Assuming for the present that this is not the case, then at Step 106 the two sets S y are 
20 compared. If the difference between the value of an element x ! of the set S x and the value of the 
correspondingly positioned element x f of the set S y is less than a threshold 8, i.e. 

Kx')-&(y')|«? 

then the two elements are considered to match. The two sets are considered to match only 
if every element x f of the set S x matches the corresponding element y f of the set S r 

25 If the sets do not match, the element jr is discarded as a candidate for further comparisons 

and the process resumes at step 102 with a fresh point % being chosen. If on the other hand, they 
match, then y_ is retained for a further iteration. Ignoring Step 107, for the present, the process 
proceeds with step 108 where the score Vis incremented. Different neighbourhood sets are then 
generated at Step 103 and the process repeated from that point. 
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When, at Step 105, the count tries of the number of comparisons exceeds a threshold, the 
process for the current x is terminated: at Step 109 the value of V is stored in the memory 6 and 
then, at step 110, a check is performed as to whether all points x have been dealt with. If so, then 
firstly the average score Va (being the average of all the individual scores V for the picture 
5 elements x in image A) is calculated at Step 1 1 1 and the process terminated at 1 12; otherwise the 
whole process is repeated from Step 100 for a fresh point x. 

The threshold value used at Step 105 could be set to a desired number of comparisons t, 
and Step 107 omitted. In practice, however, we find that the early comparisons often do not find a 
match, but once a match is found a significant number of different neighbourhood sets show 

1 0 matches with the same j. In the interests of providing a score V which is easily interpreted, the 
first few iterations are not scored; the process as shown includes a test 107 which prevents the 
score being incremented during the first init comparisons. The threshold used at Step 105 is thus 
increased by init so that the maximum potential score is t. In this way, we find that the chances of 
obtaining a high or maximum score for images which are very similar or identical is greatly 

1 5 increased. 

Specific values for the various parameter used in tests were as follows: 
Image A size xmaxi x xmax 2 ; 20x20 
Image B size xmaxj x xmax 2 : 20 x 20 
Neighbourhood parameter e: 4 
20 Neighbourhood set size m: 3 
Maximum A - B shift s: 1 
Number of comparisons t\ 50 
Initial comparisons init: 20 

Naturally, for higher definition images, correspondingly larger values of e and s would be 
25 chosen. The values of t and init have to be increased as s increases in order to obtain equally 
reliable and statistically significant results. 

The results of these tests are given later; where fee parameters used differ from those 
quoted above, this is noted. 

It will be seen that the score of a picture element x in the pattern A is high if many 
30 randomly selected S x match S y for a given y in pattern B, It is worth noting that the Visual 
Attention score (discussed in our earlier patent applications) of picture element x in pattern A is 
high if S x raw-matches S y where y is not held but is randomly selected from pattern A. Whereas 
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Visual Attention requires no memory save the single image A, the present method requires also the 
information contained in pattern B to detect commonality. 

A location x will be worthy of cognitive attention if a sequence of t neighbourhood sets S x 
matches a high proportion of the same neighbourhoods surrounding some y in pattern B. In Figure 
5 2, m = 3 picture elements x' are selected in the neighbourhood of a picture element x in pattern A 
and matched with 3 picture elements in the neighbourhood of picture element jr in pattern B. Each 
of the picture elements might possess three colour intensities, so a = and the 

neighbourhood set of the second picture element y matches the first if the colour intensities of all m 
+ 1 corresponding picture element have values within 5 of each other. Picture elements x in A that 
1 0 achieve large numbers of matches over a range of t neighbourhood sets S x with neighbourhood sets 
S y around y in B are assigned a high score. This means that neighbourhood sets in A possessing 
structure present in B will be assigned high scores. 

Some image analysis techniques carry out comparison calculations between images using 
patches that are neighbourhoods in which all the picture elements are employed. Patches match 
1 5 when a measure of correlation exceeds a certain threshold. This approach is unable to make best 
use of detail that is smaller than the size of the patch except in the case in which the correlation 
measure is designed to identify a specific texture. The random picture element neighbourhoods S x 
used in this method do not suffer from this disadvantage. 

The gain of the scoring mechanism is increased significantly by retaining the picture 
20 element location y if a match is detected, and re-using y for comparison with the next of the / 
neighbourhood sets. It is likely that if a matching picture element configuration is generated, other 
configurations will match again at the same point, and this location y_ once found and re-used, will 
accelerate the rise of the score provided that the sequence is not subsequently interrupted by a mis- 
match. 

25 If, however, S x subsequently mis-matches at that location, the score is not incremented, and 

an entirely new location y ^ pattern B is randomly selected ready for the next comparison. In this 
way competing locations in pattern B are selected against if they contain little commonality with 
the neighbourhood of x in pattern A. 

Some possible variations will now be discussed. 

30 a) It is not essential that all picture elements x be analysed: thus, if desired, a subset - 

perhaps on a regular grid - could be chosen. 

b) The above method assumes single (brightness) values for the picture elements. In 
the case of colour images, the images could firstly be converted to monochrome. Alternatively, if 
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each picture element is expressed as three values, such as red, green and blue, or luminance plus 
colour difference, then the test for a match would involve consideration of the distance between the 
element values in 3 dimensions. For example, if the colour component values are a = (a n a & a*), b 
= (b n bg, b b ) for the two image then the criterion for a match might be (a r -b r ) < 8 and I a g -b g | <5 
5 and \a b -b b \<5. See below for further discussion of distance measures. 

c) The strategy of choosing y at random could be modified by providing that when a 
element x = (x u x 2 ) produces a high score V against element & and an adjacent element x (e.g. (x x 
+1, ;c 2 )) is to be dealt with, then the first element y to be selected in image B could be the 
correspondingly adjacent element (i.e. (yi+1, j^)). Further selections of y would be random, as 

1 0 before. In the event that the high score for element x was produced as a result of comparisons with 
more than one y, then one would choose the element adjacent to whichever y made the largest 
contribution to the high score. To implement this is would be necessary to track, temporarily, the 
coordinates of the different picture elements y and their partial scores. 

d) The process need not necessarily be carried out for the whole image. For example, if a - 
1 5 region of image A has been identified as being of special interest - perhaps using the method 

described in one of our earlier patent applications referred to above - then the picture elements x 
dealt with may be just those lying within the identified region. 

References here to random selection include the possibility of selection using a pseudo- 
random process. 

20 The method is not applicable solely to images, or even three-dimensional patterns, but can 

be used for any ordered set of values in 1, 2, 3 or more dimensions. 

More generally, suppose that one wishes to generate a measure of the similarity between 
two patterns A and B, where both the patterns and the individual pattern of values within a pattern 
may have 1, 2 or more dimensions. 

25 Let a set of measurements a on pattern A correspond to a location x in A in bounded n- 

space (x h x 2 , x 2t ..., x n ) where 

x~(xi,X2,X3» ...,*«) and a = (a u a 2 ,a^ ... , a p ) 

Define a function F such that a = F(x) wherever a exists. It is important to note that no 
assumptions are made about the nature of F e.g. continuity. It is assumed that x exists if a exists. 

30 Consider a neighbourhood N of x where 

{x'eNiff |Xi - x f i| < 6i V i} 
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Select a set of rn + 1 random points S x in N where 
S x = fe'b £2,2* x^} and F(x ! ,)is defined. 

- where however one of the x' is preferably constrained to equal x. It is not found in 
practice to be necessary to constrain the random selection to avoid selection of the same point 
5 twice: i.e. the x'j need not be distinct. Indeed, the x'j could all be coincident with x or x'j . 

Select a location £ corresponding to the set of measurements b on pattern B for which F is 

defined. 

Define the set S y = {£ u y'o, j^, x ' m } where 
5 - x\ = y. " X'i a* 1 * 1 ICx'O exists. 
1 0 The neighbourhood set S x of x is said to match that of y if 

|Fj(x) - Fj(i)| < 5j and |Fj(x'0 - FjtfOI < 6j V i j. 

In general 8j is not a constant and will be dependent upon the measurements under 
comparison i.e. 

8j = fj(T(x),F(Y)) j=l...p 

1 5 Note that for some problems rather than defining a match as requiring the individual 

components aj=Fj( ) and bf=Fj( ) to match one might prefer to apply a threshold to the vector 
distance between a and b such as the city block distance. 



20 



or the Euclidean distance 



The operation of the method described in the flowchart of Figure 3 will be illustrated by 
the following test results. 

In the tests, 20 x 20 black and white images were used (i.e. the values of a and b were 
25 always either .0 or 1). The tests however differ from the earlier description in that only black 
picture elements are scored - that is, the match criterion is not merely that | a-b | <6 but that a=b=l . 
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In each case the accompanying Figure shows a first image A, plus several second images B 
with which the first was compared, marked Bl, B2 etc. (in once case two first images Al, A2 are 
compared with one image B) . The coordinates x h yi are measured from left to right and the 
coordinates x 2 , y 2 from top to bottom. Each of the second images is marked with the average score 
5 Va. Some Figures show a three-dimensional plot of the individual scores Fover the image area: in 
each case the right-hand axis represents the horizontal axis of the mark and the front axis represents 
the vertical axis of the image. The vertical axis of the plot represents the value of V (in the 
embedded recognition case of Example 2, and 50-Fin the other cases). 

Example 1. Optical Character Recognition (Figure 4) 

1 0 An image A of the letter B is compared with images B 1 - B5, of the letters A, B, C, D and 

E. The rank ordered average scores Va correspond to B, D, E, C, A. The results were generated 
with *=50, s=3 and 6 = 2. 

Example 2. Embedded Recognition (Figure 5) 

An image A of an 'X* shape is compared with another image Bl containing 9 smaller 
1 5 shapes including an 'X\ The 'X' shape is again compared with image B2 with the same 9 shapes 
except for the small 'X' which is altered. The average scores V a are 27.5 and 11.9 respectively, 
indicating the presence of £ X' features in the first and less in the second. The individual scores V 
for the respective comparisons are shown in the charts. The results were generated with t=50, s=7 
and 6 = 2. A more complicated example of embedded recognition is the identification of known 
20 faces in a crowd of people. 

Example 3. Trade mark Retrieval (Figure 6) 

An image A of a special cross shape is first compared with 4 basic shapes B 1-B4 including 
a large cross, each basic pattern representing clusters of similar shapes with high scores between 
each other. The large cross generates the largest average score V a (43.6). The same special cross 
25 in then compared with 4 varieties of cross B5-B8 including a small version of itself (B5) which 
obtains a maximum score of 50. The chart provides the values of (50 - V) for each picture element 
in the comparison with the large cross - the tall columns indicate the disparity with the ends of the 
lower legs of the cross. The results were generated with /=50, 5=7 and e = 2. 

Example 4. Fingerprint Recognition (Figure 7) 

30 More complex patterns, some possessing a great deal of similar material, are able to be 

distinguished and retrieved as in the trade mark example. A simplified arch A matches best with a 
representative arch B4 (49.3), and then with an identical pattern (49.9) amongst other arches. The 
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chart provides the values of (50 - V) for each picture element in the comparison with the 
representative arch pattern - the tall columns indicate the disparity with the ridge ends and on the 
border. The results were generated with *=50, s=3 and e = 2. 

Example 5. Face Recognition (Figure 8) 

5 Although the data is extremely simplified, this example illustrates an implementation of 

face recognition. A face A is matched with 4 representative faces Bl - B4 and then with 4 faces in 
a sub-cluster B5 - B8 represented by the best fitting representative face B2 (45.7). The best fit is 
with a non-identical face B7 in the sub-cluster (49.6). The chart provides the values of (50 - V) for 
each picture element in the comparison with the 2 nd closest pattern B6 (48.2) - the tall columns 
1 0 indicate the disparity in the mouth region. The results were generated with /=50, r=3 and e = 2. 

Example 6. Surveillance (Figure 9) 

This example illustrates how intruders might be detected against a variable background. A 
pattern Al representing a moving cloud, tree and fence obtains a high average score V a (48.0) 
against a standard background B, but a version (A2) with an intruder present obtains a much lower 
1 5 score (33.1). The chart provides the values of (50 - V) for each picture element in the intruder 
image (33.1) - the tallest columns indicate the location of the intruder. The results were generated 
with /=50, s=3 and s = 2. 

This method carries out an analysis of two images that identifies perceptually significant 
common features and differences without being affected by scale differences, and by local relative 

20 distortions and translations. This means that it can be applied to recognition problems in which the 
object to be recognised is embedded in another image. It also means that images may be 
categorised according to the measure of feature similarity with each other and these values used to 
define a multi-class recognition system and a Query By Example retrieval system. By the same 
token the method may be used to detect dissimilarities as in the case of intruder detection or forgery 

25 detection. It may be used to detect disparities between images to detect motion or parallax. It does 
not necessarily rely upon prior knowledge of the content of the images and no training process is 
necessary. The algorithm is eminently suitable for parallel implementatiop. 

Various advantages of the methods we have described will now be discussed. 

Pattern Recognition 

30 Standard approaches to pattern recognition require large numbers of representative patterns 

from all classes in order to satisfy the requirements of the training process for the classifier. Such 
classifiers will still fail if the features selected do not characterise unseen patterns not represented 
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properly in the training set. This method requires no training set, save the two images under 
comparison. 

It is normal for pattern recognition techniques to make special provision for scale 
differences between the patterns being classified. Such differences in the visual domain can arise 
5 because of perspective and other factors and it is common for prior knowledge to be incorporated in 
the classifier to compensate. This is not necessary in this method, provided that e is not too large. 

Another advantage is the ability to identify patterns embedded within larger data 
structures without suffering from exponentially growing computational demands or the effects of 
noise. This method therefore has application to the problem of detecting copyright infringement 

1 0 where portions of material have been cropped from larger works (of art, for example), and the task 
of reducing the size of databases where it is known that duplication is prevalent. It also has direct 
application to the problem of identifying scene-of-crime fingerprints where only a part of the total 
print is available for matching. In the case of facial recognition the method lends itself to searches 
based upon restricted portions of the unknown face. This means, for example, that searches can be 

1 5 carried out purely on the basis of the eye and nose region in cases where beards and moustaches 
might lead to ambiguity. 

Unless provision is made for specific pattern content, the standard template approaches to 
pattern recognition fail when the patterns under comparison differ because of local distortions or 
small movements, as would be the visual case with rustling trees, moving clouds, changes in facial 
20 expression, scene-of-crime fingerprints on irregular surfaces, or noise, for example. Such provision 
requires prior knowledge of the application and will still cause the system to fail if the unseen 
pattern distortions do not conform to the system design requirements. This method is able to ignore 
the effects of local distortions without prior knowledge of the type of distortion. 

The method may be applied to patterns of any dimension, such as one-dimensional audio 
25 signals, three dimensional video data (x,y,time), or n-dimensional time dependent vectors derived 
from any source such as sensor arrays. In the case of speech recognition it is able to handle 
variations in the speed of the speech without the use of special heuristics. A conventional approach 
uses Dynamic Time Warping to overcome this problem, but invokes greater computational effort 
and the danger of the warping process leading to increased misclassifications especially in a large 
30 multi-class problem. Furthermore portions of utterances would be sufficient for word identification 
using this invention if they were unique in the domain of discourse (e.g. 'yeah' instead of 'yes', 
missing the final sibilant). 
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This method deals with these problems by measuring the score of neighbourhoods in each 
image. Whereas a picture element is important within a single image if its neighbourhood is 
significantly different from most other parts of the same image (as discussed in our earlier patent 
applications mentioned above), it is cognitively important if its neighbourhood is significantly 
5 similar to that in a neighbourhood in the second image. The effects of mis-registration and local 
distortions are considerably reduced as the scores are not affected within certain limits. 

The scores can be used to cluster groups of patterns possessing high scores relative to each 
other. Representative patterns taken from each cluster may themselves be clustered to form super- 
clusters and the process continued to structure very large pattern databases. Query-By-Example 

1 0 retrieval can be carried out by measuring the scores to each of the top level representative patterns 
and then to each of the representative patterns in sub-clusters corresponding to the highest scoring 
representative pattern in the previous cluster. It is likely that some representative patterns will 
represent overlapping clusters containing patterns which happen to possess high scores with those 
representative patterns. The method may be used in combination with a visual attention mechanism 

1 5 to rapidly define an area of interest in the pattern and then derive scores between this restricted area 
and a set of reference patterns (B patterns). In an aviation context a flying object might be quickly 
detected using visual attention and subsequently computed scores would reveal whether the object 
was likely to be a bird or a plane. 

Disparity Detection 

20 Standard approaches to disparity detection rely heavily upon accurate registration between 

two images so that the subtraction (which may be carried out piecewise for a small areas of the total 
image) takes place between picture elements that correspond to the same points on the original 
object pictured in the two images. The resulting difference-image highlights those areas that 
correspond to differences in the original images. This becomes extremely difficult if noise is 

25 present as uncertainty is introduced into the estimate of the correct registration position and many 
spurious differences can be generated as a result. 

Even if noise is absent local distortions or slight subject movements will cause mis- 
registration and areas of difference will be highlighted which are of little interest unless the 
distortion or movement itself is being measured. Linear or non-linear digital image registration 
30 techniques prior to subtraction partially compensates but does not eliminate this problem in a large 
proportion of cases [4]. 

A severe problem faced by conventional methods arises from the different conditions under 
which the two images were created. Such differences may stem from the lighting, weather 
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conditions, a slightly different viewpoint, a different make of film, or a different camera with a 
different optical system. These interferences all contribute towards the generation of spurious 
differences between the images that do not relate to the content. 

Differences between the two images caused by the conditions of image capture will be 
5 factored out by this method because such differences will affect the score equally across all parts of 
the image and will not disturb the rank ordering of the individual scores. Significant illumination 
differences between the two images are compensated by employing a larger threshold for the 
picture element matching (8j see above). This would be necessary, for example, if X-ray exposure 
times were different between the two images. 

1 0 This method also has the advantage of being able to detect multiple disparities in which 

image A is compared with images B132JB3 etc. This would be useful in the case of CCTV 
intruder detection in which image frames B132,B3 etc would be typical examples of different 
atmospheric conditions and other normal background states, and an alarm would be raised only if 
disparities were detected in all the normal image frames. 

15 It will be seen that these methods have relevance to almost all applications requiring pattern 

recognition: 

Face recognition, fingerprint recognition, OCR, image retrieval, trademark identification, 
forgery detection, surveillance, medical diagnosis, and others. 

They are particularly relevant to the medical industry where the comparison of very 
20 variable and noisy images is necessary to track the progress of medical conditions. For example, 
the identification of likely cancerous growth in mammograms is often gleaned from X-rays taken at 
different times and any automatic assistance would increase radiologist's throughput. In another 
example, contrast enhancement is normally obtained by subtracting the X-ray image of the normal 
breast from one taken after contrast injection and obtaining proper registration is not only difficult 
25 to obtain but is critical to the success of the process. The methods we have described can provide a 
clearer result without the attendant registration problems. 

Advanced software detection systems for mammograms (eg. http://www.r2tech.com/prd/) 
are quite capable of spotting specific features commonly associated with cancer (eg 
microcalcifications), but the problem of temporal comparisons is largely not addressed. 

30 References 

[1] Vailaya A et al., Image Classification for Content-Based Indexing, IEEE Trans on 
Image Processing, Vol 10, No 1, pp 1 17 - 130, Jan 2001. 



WO 03/081523 



PCT/GB03/01209 



-14- 

[2] Santini S & Jain R, Similarity Matching, in Proc 2" d Asian Conf on Computer Vision, 
pages H 544-548, IEEE, 1995. 

[3] IEEE Trans PAM - Special Section on Video Surveillance, vol 22 No 8, Aug 2000. 

[4] Brown L G, A survey of image registration techniques, ACM Computing Surveys, 
5 Vol.24, No. 4 (Dec. 1992), pp. 325-376. 



WO 03/081523 



PCT/GB03/01209 



-15- 
CLAIMS 

1. A method of comparing a first pattern represented by a first ordered set of elements each 

having a value with a second pattern represented by a second ordered set of element each having a 
5 value, comprising performing, for each of a plurality of elements of the first ordered set the steps 
of: 

(i) selecting from the first ordered set a plurality of elements in the vicinity of the element 
under consideration, the selected elements having, within the ordered set, respective mutually 
different positions relative to the element under consideration; 

1 0 (ii) selecting an element of the second ordered set; 

(iii) comparing the selected plurality of elements of the first ordered set with a like 
plurality of elements of the second ordered set each of which has within the second ordered set the 
same position relative to the selected element of the second ordered set as a respective one of the 
selected plurality of elements of the first ordered set has relative to the element under 
1 5 consideration, said comparison comprising comparing the value of each of the selected plurality of 
elements of the first ordered set with the value of the correspondingly positioned element of the 
like plurality of elements of the second ordered set in accordance with a predetermined match 
criterion to produce a decision that the plurality of elements of the first ordered set matches the 
plurality of elements of the second ordered set; 

20 (iv) repeating said comparison with a fresh selection of the plurality of elements of the first 

ordered set and/or a fresh selection of an element of the second ordered set; and 

(v) generating for the element under consideration a similarity measure as a function of the 
number of comparisons for which the comparison indicates a match. 

25 2. A method according to claim 1 in which the plurality of elements of the first ordered set 

includes the element under consideration. 

3. A method according to claim 1 or 2 in which the selection of the plurality of elements of 

the first ordered set is random or pseudo-random. 



30 
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4. A method according to claim 1 ,2 or 3 in which the selection of an element of the second 
ordered set is random or pseudo-random. 

5 . A method according to any one of the preceding claims in which, following a comparison 
5 resulting in a match decision, a further comparison is performed with a fresh selection of the 

plurality of elements of the first ordered set and the same selection of an element of the second 
ordered set. 

6. A method according to any one of the preceding claims in which, following a comparison 
1 0 resulting in a decision that there is no match, a further comparison is performed with a fresh 

selection of an element of the second ordered set and the same selection of the plurality of elements 
of the first ordered set. 

7. A method according to any one of the preceding claims, in which each value comprises a 
1 5 plurality of components. 

8. A method according to any one of the preceding claims, in which the comparison is 
performed a predetermined number of times. 

20 9. A method according to any one of the preceding claims, in which the similarity measure is 
a function of the number of comparisons for which the comparison indicates a match, excluding an 
initial number of comparisons. 

10. A method according to any one of the preceding claims in which, when an element under 
25 consideration has resulted in a similarity measure indicative of a large number of matches and a 
further element of the first ordered set is to be considered, the first selection of an element of the 
second ordered set for comparison therewith is of that element of the second ordered set whose 
position, within the second ordered set, relative to an element of the second ordered set giving rise 
to said large number of matches, is the same as the position of the further element of the first 
30 ordered set relative to the previous element under consideration. 
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